StreamFlow: Cross-Breeding Cloud With HPC

نویسندگان

چکیده

Workflows are among the most commonly used tools in a variety of execution environments. Many them target specific environment; few make it possible to execute an entire workflow different environments, e.g., Kubernetes and batch clusters. We present novel approach execution, called StreamFlow, that complements graph with declarative description potentially complex makes onto multiple sites not sharing common data space. StreamFlow is then exemplified on bioinformatics pipeline for single-cell transcriptomic analysis workflow.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Hpc Cloud Applied to Lattice Optimization∗

As Cloud services gain in popularity for enterprise use, vendors are now turning their focus towards providing cloud services suitable for scientific computing. Recently, Amazon Elastic Compute Cloud (EC2) introduced the new Cluster Compute Instances (CCI), a new instance type specifically designed for High Performance Computing (HPC) applications. At Berkeley Lab, the physicists at the Advance...

متن کامل

Analyzing Performance and Efficiency of HPC Applications in the Cloud

Elasticity is a key feature of cloud computing in which the resources can be dynamically changed on-the-fly. In the HPC scenario, a user may want to adjust the resources availability to increase performance. A way to measure performance and efficiency of parallel applications with fixed number of resources is the speedup metric. However this metric does not apply to elastic cloud infrastructure...

متن کامل

Scheduling Concurrent Workflows in HPC Cloud through Exploiting Schedule Gaps

Many large-scale scientific applications are usually constructed as workflows due to large amounts of interrelated computation and communication. Workflow scheduling has long been a research topic in parallel and distributed computing. However, most previous research focuses on single workflow scheduling. As cloud computing emerges, users can now have easy access to on-demand high performance c...

متن کامل

Performance Evaluation of Multiple Cloud Data Centers Allocations for HPC

This paper evaluates the behavior of the Microsoft Azure G5 cloud instance type over multiple Data Centers. The purpose is to identify if there are major differences between them and to help the users choose the best option for their needs. Our results show that there are differences in the network level for the same instance type in different locations and inside the same location at different...

متن کامل

Energy-Efficient Scheduling of HPC Applications in Cloud Computing Environments

The use of High Performance Computing (HPC) in commercial and consumer IT applications is becoming popular. They need the ability to gain rapid and scalable access to high-end computing capabilities. Cloud computing promises to deliver such a computing infrastructure using data centers so that HPC users can access applications and data from a Cloud anywhere in the world on demand and pay based ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: IEEE Transactions on Emerging Topics in Computing

سال: 2021

ISSN: ['2168-6750', '2376-4562']

DOI: https://doi.org/10.1109/tetc.2020.3019202